Papers with zero-shot cross-lingual setting
LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Tagging (2022.coling-1)
Copied to clipboard
| Challenge: | LINGUIST generates annotated data for Intent Classification and Slot Tagging (IC+ST) we demonstrate fine-tuning of a large-scale seq2seq model to control outputs of multilingual data generation. |
| Approach: | They propose a method for generating annotated data for Intent Classification and Slot Tagging (IC+ST) they use a 5-billion-parameter multilingual sequence-to-sequence model to fine-tune it on a flexible instruction prompt. |
| Outcome: | The proposed method outperforms state-of-the-art approaches on a SNIPS intent setting and shows significant improvement on IC+ST in a cross-lingual setting. |
Multilingual Coreference Resolution in Multiparty Dialogue (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing datasets for entity coreference resolution are limited to English and other languages are rare. |
| Approach: | They propose to use TV transcripts to create multilingual multiparty coreference datasets that leverage existing subtitles in Chinese and Farsi. |
| Outcome: | The proposed dataset re-annotates for coreference on TV transcripts and then leverages existing subtitle translations to create a multilingual corpus. |
English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Too (2020.aacl-main)
Copied to clipboard
Jason Phang, Iacer Calixto, Phu Mon Htut, Yada Pruksachatkun, Haokun Liu, Clara Vania, Katharina Kann, Samuel R. Bowman
| Challenge: | a study of intermediate-task training in monolingual English shows that it improves model performance on non-English language understanding tasks. |
| Approach: | They evaluate whether English intermediate-task training is still helpful on non-English target tasks . BUCC and Tatoeba sentence retrieval tasks see large improvements . |
| Outcome: | The proposed model outperforms existing models on non-English language understanding tasks. |
Robustification of Multilingual Language Models to Real-world Noise in Crosslingual Zero-shot Settings with Robust Contrastive Pretraining (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies on robustness of pretrained multilingual models are limited to the English language. |
| Approach: | They propose to use data augmentation and contrastive loss term to boost robustness of multilingual models in cross-lingual settings. |
| Outcome: | The proposed model outperforms existing models on clean and noisy data in the cross-lingual setting. |
Hybrid Knowledge Transfer for Improved Cross-Lingual Event Detection via Hierarchical Sample Selection (2023.acl-long)
Copied to clipboard
| Challenge: | Recent efforts to train cross-lingual models on source language fail to take advantage of data transfer . current methods focus on learning task-specific information from syntactical features or word-label relations in target language. |
| Approach: | They propose a hybrid knowledge-transfer approach that leverages a teacher-student framework . the model is evaluated on a distinct target language for which there is no labeled data . |
| Outcome: | The proposed model achieves state-of-the-art results on 9 morphologically-diverse target languages across 3 distinct datasets. |
On the Cross-lingual Transferability of Monolingual Representations (2020.acl-main)
Copied to clipboard
| Challenge: | State-of-the-art unsupervised multilingual models generalize in zero-shot cross-lingual setting . generalization ability attributed to shared subword vocabulary and joint training across multiple languages . |
| Approach: | They propose an approach that transfers a monolingual model to new languages at the lexical level. |
| Outcome: | The proposed approach is competitive with multilingual BERT on cross-lingual classification benchmarks and on a new cross-linguistic question answering dataset. |
Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual Prompt (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing work focuses on monolingual prompts, but we study multilingual prompt for multilingual models. |
| Approach: | They propose a model that uses a unified prompt for all languages, called UniPrompt, to alleviate the effort of designing different prompts for multiple languages. |
| Outcome: | The proposed model outperforms baseline models in the zero-shot cross-lingual setting. |